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ABSTRACT. Motivated by generalized forms of the exponential and hyperbolic functions, 


we introduce a new generalization of the sigmoid function and further study some of its 


analytical properties. 


1. INTRODUCTION 


The sigmoid function, which is otherwise known as the standard logistic function is defined 














for z € R = (—co, oo) as 
e 1 


(1) oo 1+ e ~ l+e-? 


(2) = : + Š tanh (5) 


It is monotone increasing and its first derivative is bell-shaped. See the plots below. 





This special function is applied in many scientific disciplines such as machine learning, 
artificial neural networks, computer graphics, medicine, probability and statistics, biology, 
ecology, population dynamics, demography, and mathematical psychology. See for instance 
[1], [2], [8], [4], [5], [6], [8], [10], [11], [12], [13], and the several related references therein. 
Due to the important roles of this function, recent works have focused on investigating 
its analytical properties. In [9], the authors studied properties of the function such as 
starlikeness and convexity in a unit disc. In [14], the author studied among other things 
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FIGURE 1. Plot of S(z) FIGURE 2. Plot of S’(z) 


some properties such as subadditivity, log-concavity, monotonicity and geometric concavity 
of the function. Thereafter, in [17], the authors introduced a generalization of the function 
and by adopting the techniques of [14], they proved analogous properties for the generalized 
function. They also considered some statistical properties of the generalized function. 

Motivated by generalized forms of the exponential and hyperbolic functions, the goal of 
this paper is to introduce a new generalization of the sigmoid function and to further study 
some of its analytical properties. 




















Definition 1.1 ([16]). Let 6:1 C R? = (0,00) > Rt. Then ¢ is said to be geometrically 


convex on I if 


(3) o(2"y") < lN eN” 


for all x,y € I and r € [0,1]. If the inequality in (3) is reversed, then ¢ is said to be 



































geometrically concave on I. 





























Lemma 1.2 ([15]). Let 6:1 C R* > R? be a differentiable function. Then the following 
statements are equivalent. 


(a) The function @ is geometrically convex (concave). 





(b) The function oa is increasing (decreasing). 
2. GENERALIZED SIGMOID FUNCTION AND ITS PROPERTIES 


In this section, we introduce a new generalization of the sigmoid function and further 
study some of its properties. We begin with the following auxiliary definitions. 
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Definition 2.1. Let a > 1. Then the generalized exponential function is defined as 


= (Ina)”z” 
4 tay e 
(4) G are 











for all z€ R. 





From this definition, we define the generalized hyperbolic cosine, hyperbolic sine and 
hyperbolic tangent functions as follows. 


Definition 2.2. Let a > 1. Then the generalized hyperbolic cosine, hyperbolic sine and 
hyperbolic tangent functions are respectively defined as 








(5) cosh,(z) = EE, 
(6) sinh,(z) = a ; 
2 
sinh,(z) a*—a* 2 
7 t ha = — = 
(7) ankal) cosha(z)  a?+a-? 1 +a” 
for all z € R. 











These generalized functions satisfy the following identities. 


(8) cosh,(z) + sinh,(z) = a’, 

(9) cosh,(z) — sinha (2) = a~, 

(10) (cosh,(z))’ = (Ina) sinh,(z), 

(11) (sinh, (z))" = (In a) cosha(z), 

(12) (tanba(2))! = Ps. 

(13) (cosha(z))” + (sinha(z))” = (Ina)’a*, 
(14) (cosh,(z))” — (sinha(z))” = (Ina)?a~*, 
(15) cosh?(z) + sinh?(z) = cosh,(2z), 
(16) cosh?(z) — sinh?(z) = 1. 


Several other identities can be derived from (5), (6) and (7). In particular, if a = e, where 
e = 2.71828... is the Euler’s number, then the above definitions and indentities reduce to 
their ordinary counterparts. Motivated by the above definitions, we defined a generalization 
of the sigmoid function as follows. 
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Definition 2.3. Let a > 1. Then the generalized sigmoid function is defined as 


z 


a 
1 a rt are Treaty 
(17) Sa(z) moe 
1 1 zZ 
(18) = 3 + gtanhe (5). 














for all z € R. Clearly, when a = e, then S,(z) reduces to S(z). 


Direct computations reveals that this special function satisfies the following properties. 


zZ 








1 _ a = = 

(19) Sile) = (na) = (In) Sa(2) (1 ~ Sala), 
2a*(1 — a7) 2 
(20) 5. (z) = (na) (ita) = (Ina)*S,(z)(1 — Salz) — 2S, (z)), 
(21) (In S,(z))" + n a) S! (z) = 0, 
(22) Salz) = 1 — Sa(—z), 
(23) Sa(2) = (Ina)Sa(z)Sa(—2), 
(24) lim 5,2) = 1, 
25 lim S = 
( ) PENT alz) —_ 2? 
(26) lim Su(z) = 0, 
(27) lim S,(z) = 0, 
28 ime 
(28) Pare a(z) = A 
_ In(1 +a’) 

(29) IEG dz = a + k; 
where k is a constant of integration. The function ae) is a generalization of the softplus 


function which was first defined in [7]. 


Remark 2.4. Figure 3 and Figure 4 respectively show the plots of the generalized sigmoid 
function and its derivative for some particular values of a. It is evident that the shapes of 
Ina 


these functions depend on the values of a. The maximum value of S} (z) is 3% for any a > 1. 


Remark 2.5. The function S,(z) is a cumulative distribution function in the sense that: 
(a) lim,- Salz) = 0, 
(b) limno Salz) = 1, 


(c) Salz) is monotone increasing, 
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FIGURE 3. Plot of S,(z) FIGURE 4. Plot of S! (z) 
(d) S,(z) is right-continuous. 
The function S% (z) is the associated probability density function. 
Theorem 2.6. The generalized sigmoid function satisfies the inequality 
(30) Sa(w +2) < Salw) + Salz), 


for all w,z E€ R. That is, Sa(z) is subadditive on R. 

















Proof. Let (t) = iq where t > 0. Then 4(t) is strictly increasing for all t > 0. Also, let 
w,z € R. Then, 














a” a? aY +a? + 2a” = 
Sa 5,(2)= | = 
(w) = (2) 1 +a” 1 + a? 1 + aY +a” + aYt? 


a” + az + avt? 


























> 

1 + aY +a? + att? 
— (a” A a” ae qr) 
> v(a"**) 
= Salw + z), 











which gives the desired result. 





Theorem 2.7. The generalized sigmoid function satisfies the inequality 


(31) Sa (Sw + (1 — 4)z) > [Saw] S], 








for all w,z € R and ô € [0,1]. That is, Sa(z) is logarithmically concave on R. 











Proof. It suffices to show that (In.$,(z))” < 0 for all z € R. Let z € R. Then 


(In $,(2))" = (3) _ SASA) - (SQ)? _ _ (naja 











AOE STe =" 
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which concludes the proof. 





Remark 2.8. Theorem 2.7 is equivalent to the following statements. 














(a) Sae) Sal) < (Sele ))? for all z € R. 
(b) 3 i R. 











Corollary 2.9. The inequality 


(32) 














tod 


is satisfied for all x € 





Proof. By letting w = x — 1, z = xz + 1 and ô = i in Theorem 2.7, we obtain 
[Sa(x)]” > Salz — 1)Sa(a + 1), 











which yields inequality (32). 





Theorem 2.10. The generalized sigmoid function satisfies the inequality 
1-A dr 
(33) a < [Sat + DP <1, 
l+a Sal Az + 1) 
for z € (0,00) and A > 1. The inequality is reversed if 0< A <1. 


Proof. For z € (0,00) and À > 1, let B(z) = Ke6 and A(z) = In (2). Then 














Sa 
i Salz +1) Saz +1) 
h'(z) =A a 0 
= E GEFI LAF” 
since i (z) is increasing. Then for z € (0,00), we have 


f(0) < A(z) < lim A(z) = 1, 


ZOO 


which yields inequality (33). The proof for the case where 0 < A < 1, follows a similar 











procedure. Hence we omit the details. 





Theorem 2.11. For a € [0,1], the inequalities 








(34) l aioe ea e 
(35) iesea- =2, ze (-00,0), 
are satisfied. 
Proof. Note that 

(S) = SLO = (map EE > (o0 
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respectively if z E€ (—oo,0) or z € (0,00). Thus, S’(z) is increasing for z € (—oo,0) and 
decreasing for z € (0,00). Let a € [0,1] and ¢(z) = Salaz) — a Salz). Now suppose that 
z € (0,00). Then az < z and then 

p(z) = a [Si (az) — S,(z)] > 0, 


since S’(z) is decreasing for z € (0,00). Hence $(z) is increasing for z € (0,00). Therefore 


l-a 





= ¢(0) < A(z) < lim d(z) =1—-a, 


2 ZO 


which gives (34). Next suppose that z € (—oo,0). Then az > z and 
(2) = a [Sa (az) — Sa(2)] > 0, 


since S/ (z) is increasing for z € (—oo,0). Hence ¢(z) is increasing for z € (—oo,0) as well. 
Consequently 
l-a 


0= lim ¢(z) < ¢(z2) < 6(0) = , 


zZ—>—00 2 

















which gives (35). 


Remark 2.12. Inequalities (34) and (35) imply that S,(az) > aS,(z) for all z € (—oo, 00) 
and a € [0,1]. In other words, the function S,(z) is -+-oo-star-shaped. 
Theorem 2.13. The inequality 


In(1 + a*) — In(1 + a”) P 
Z-2 


(36) (In a)S,(az) < 





(Ina)Sa(z), 
holds forO<a<z 


Proof. Let 0 < x < z and consider the function y(t) = In(1 + a‘) on the interval (x, z). By 
the mean value theorem, there exist a c € (x, z) such that 
In(1 + a”) — In(1 + a”) 
zZ-2£ 


= y (o). 


Since y'(t) = (In a)Sa(t) is increasing, then for c € (x, z), we have 





y(x) < y (e) <7 (2), 











which is equivalent to (36). 





Remark 2.14. If x = 0 in Theorem 2.13, then we obtain 


(ina) 
2 


In2+ z<In(1+a’) < ln2 + (Ina)zS,(z), z>0, 


which gives an estimate for the generalized softplus function interms of the generalized 
sigmoid function. 
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Theorem 2.15. Letr >1. Then 











Sale) \ — Balt) 
oe Gay < giro)” 
if x <y and 
INTE — [S.(x)]" ~ 1+a" 
038) G) Sada Usar 





if x € (0,1). Equality holds when r = 1. 





Proof. Let r > 1, 0(z) = Be@F and g(x) = In (x). Then 


so =r T Ra °° 





Thus, (x) is increasing. Then for x < y, we have 
A(x) < Oly), 


which yields (37). Similarly for x € (0,1), we have 


r-1 
La 
— = i < < l < a aN 
(3) = RE <0) < ie) = E 














which yields (38). This completes the proof. 


Remark 2.16. If 0 < r < 1 in Theorem 2.15, then we obtain the reverse cases of inequalities 
(37) and (38). 


Theorem 2.17. The inequality 


(39) Sa (ay) = [Sea ISP, 




















holds for x,y E€ R? and A € [0,1]. That is, S! (z) is geometrically concave on R*. 




















Proof. Let z € R*. Then it follows from (19) and (20) that 





and by direct computation, we obtain 


Vea k = (Ina) Fees = na) <0. 


In view of Lemma 1.2, we conclude that S! (z) is geometrically concave on R* and this is 


























equivalent to inequality (39). 
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Theorem 2.18. The inequalities 


















































Salz + 1) 
4 1 
(40) E 
Sal) 
41 z l 
(41) G e 
hold for all z E€ R. 
Proof. Let A(z) = See) for all z € R. Then 
_ ata 
Bee Ee eas 
and 
ManaSite m 
Also, 
NORE EDOD 
A(z) Salz +1) Salz) i 
since a) is decreasing. Thus, A(z) is decreasing. Hence for all z € R, we have 


1= lim A(z) = Ale) < lim A(z)=a, 


z—> 00 Z—>—O0O 


which gives inequality (40). Next we have 














Si (z) Ina 
= lim = lng, 
zZ—>—00 Salz) z=—œ ] + a” 
and 
4 
l 
lim Sal?) L lim 22 = 
zZz— 00 Salz) z= l + a? 
Then by the monotonicity of s. we obtain 
ic, O eN 








z> §,(z) Salz) 24-00 Salz) 
which gives inequality (41). 


Theorem 2.19. The inequalities 





1 Si(z+1) 
(42) A < “Se <a, 
ti 
(43) — lna < a < lna, 





hold for all z € 








ve 
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Proof. Recall that oe = (Ina) 5% for all z € R. Then 


(ZY -omari 
































which implies that ae is decreasing. Also, 
1I S" 
lim — a) =Ina and lim a(2) =-—lna 
che Sl) Me Sl) 
Hence for z € R, we have 
aon 0) DE: 











DH 
2 














which gives inequality (43). Next, let D(z) = atl for all z € R. Then 





zZ—>—00 z—>— oo 1+ atı 
and 
: l+a? \? 1 
im DG) = ma (Gh) = 


In addition, we obtain 
De _ Si(z+1)_ Sle) _ 
D(z) St(zt+1)  S!(z) l 


Thus, D(z) is decreasing. Hence for all z € R, we have 














= lim D(z) < Diz) < lim D(z)=a, 


z—>> 00 Z—>—0O 





which gives inequality (42). 











Theorem 2.20. The inequalities 


qe Sa(z + 1)Sa(y + 1) 
l+a7 S,(a+y+1) 








(44) <1, 


holds for all x,y € |0,00). 


Proof. Let A(x, y) = Stilig for all z, y € [0,00). Further let h(x, y) = ln A(x, y). Then 


ð eel. Katyt!) 
ag Mt) ~ Sala t1) Sile+y+1) av 





Thus, A(x, y) is increasing. Hence 





= lim A(z, y) <A(z,y) < lim A(z, y) = Saly +1) <1, 
1 +a x—>0 r— 00 





which gives inequality (44). 
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3. CONCLUSION 


In this work, we have introduced a new generalization of the sigmoid function which 


is frequently applied in machine learning, artificial neural networks, computer graphics, 


medicine, probability and statistics, biology, ecology, population dynamics, demography, 


and mathematical psychology. It is our fervent hope that, this new generalization and the 


properties established thereafter, will find useful applications in these scientific disciplines. 


We also anticipate that for some particular values of a, this generalized function may be 


applied in modeling datasets with heavy tails. 


N =e 
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